A Unified Approach for Modeling and Optimization of Energy, Makespan and Reliability for Scientific Workflows on Large-Scale Computing Infrastructures

Green computing has received significant attention in the past few years. Although some research has addressed cooling and energy usage reduction in large data-centers [1], they do not control how resources are used by applications. Scientific workflows are a useful representation for managing the execution of large-scale computations on high performance computing (HPC) and high throughput computing (HTC) platforms [2]. In scientific workflow applications, resource provisioning and utilization optimizations have been investigated to reduce energy consumption on Cloud infrastructures [3], [4]. However, existing research is largely limited to the measurement of energy usage according to resource utilization when running a program on an execution node. Furthermore, most existing optimization techniques for workflows are limited to single objectives (e.g. makespan), and some can deal with only two objectives. There does not exist an approach that deals with an arbitrary number of objectives and no scheduling technique explored tradeoffs among makespan, energy consumption, and reliability. We recently proposed [5] an energy consumption model for analyzing and profiling energy usage that addresses resource utilization, data movement, and I/O operations. Although our model assembles several models (computing, networking and storage systems) validated in a real execution environment, it still makes strong assumptions on the resource characteristics (e.g. single core homogeneous virtual machines), and ignores external loads. In this work, we propose 1) an extension of our energy consumption model to address real large-scale infrastructure conditions (e.g. heterogeneity, resource unavailability, external loads); 2) the validation of the model in a fully instrumented platform able to measure the actual temperature and energy consumed by computing, networking, and storage systems; and 3) a multi-objective optimization approach to explore tradeoffs among makespan, energy consumption, and reliability for multi-objective workflow scheduling.